ci: share sccache setup across workflows#811
Open
abnobdoss wants to merge 1 commit into
Open
Conversation
abnobdoss
force-pushed
the
ci/share-sccache-setup
branch
from
July 4, 2026 00:50
67d0a82 to
996a694
Compare
abnobdoss
force-pushed
the
ci/share-sccache-setup
branch
4 times, most recently
from
July 16, 2026 04:57
155e484 to
493a9e1
Compare
abnobdoss
marked this pull request as ready for review
July 16, 2026 05:41
There was a problem hiding this comment.
Pull request overview
This PR reduces duplicated sccache setup across CI workflows by introducing two composite actions (setup-sccache and save-sccache) and updating existing workflows to use them, including a more reliable Windows MSVC toolset-version-based cache key suffix resolution.
Changes:
- Added
.github/actions/setup-sccacheto configure sccache env, resolve Windows MSVC toolset key suffix, restore cache, and startmozilla-actions/sccache-action. - Added
.github/actions/save-sccacheto show sccache stats and save cache on main branch runs. - Refactored multiple workflows to pass only a cache key prefix at call sites.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/test.yml | Replaces inline sccache env/cache steps with composite setup/save actions across Ubuntu/macOS/Windows/Meson legs. |
| .github/workflows/sql_catalog_test.yml | Refactors sccache restore/save to use the shared composite actions (including Windows suffix handling). |
| .github/workflows/sanitizer_test.yml | Uses shared sccache setup/save composites instead of inline cache steps. |
| .github/workflows/cpp-linter.yml | Switches to shared sccache setup/save composites for the linter build step. |
| .github/workflows/aws_test.yml | Uses shared sccache setup/save composites for AWS test jobs. |
| .github/actions/setup-sccache/action.yml | New composite action that centralizes sccache env + restore + Windows MSVC key suffix + sccache startup. |
| .github/actions/sccache/resolve-msvc-key-suffix.ps1 | New PowerShell helper to compute MSVC toolset version suffix via vswhere + VCToolsVersion file. |
| .github/actions/save-sccache/action.yml | New composite action to show stats and save sccache cache (main branch). |
Comment on lines
+38
to
+40
| echo "SCCACHE_DIR=${{ github.workspace }}/.sccache" >> "$GITHUB_ENV" | ||
| echo "SCCACHE_CACHE_SIZE=${CACHE_SIZE}" >> "$GITHUB_ENV" | ||
| echo "SCCACHE_KEY_SUFFIX=" >> "$GITHUB_ENV" |
Comment on lines
+34
to
+36
| - name: Save sccache cache | ||
| if: ${{ github.ref == 'refs/heads/main' }} | ||
| uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 |
Comment on lines
+69
to
+72
| - name: Save sccache | ||
| uses: ./.github/actions/save-sccache | ||
| with: | ||
| path: ${{ github.workspace }}/.sccache | ||
| key: sccache-test-ubuntu-${{ matrix.cmake_build_type }}-${{ github.run_id }} | ||
| key-prefix: sccache-test-ubuntu-${{ matrix.cmake_build_type }} |
Comment on lines
+98
to
+101
| - name: Save sccache | ||
| uses: ./.github/actions/save-sccache | ||
| with: | ||
| path: ${{ github.workspace }}/.sccache | ||
| key: sccache-test-macos-${{ github.run_id }} | ||
| key-prefix: sccache-test-macos |
Comment on lines
+142
to
+145
| - name: Save sccache | ||
| uses: ./.github/actions/save-sccache | ||
| with: | ||
| path: ${{ github.workspace }}/.sccache | ||
| key: sccache-test-windows-${{ env.MSVC_VER }}-${{ github.run_id }} | ||
| key-prefix: sccache-test-windows |
Comment on lines
+199
to
+202
| - name: Save sccache | ||
| uses: ./.github/actions/save-sccache | ||
| with: | ||
| path: ${{ github.workspace }}/.sccache | ||
| key: sccache-meson-${{ matrix.runs-on }}-${{ github.run_id }} | ||
| key-prefix: sccache-meson-${{ matrix.runs-on }} |
Comment on lines
+115
to
+118
| - name: Save sccache | ||
| uses: ./.github/actions/save-sccache | ||
| with: | ||
| path: ${{ github.workspace }}/.sccache | ||
| key: sccache-sqlcatalog-${{ matrix.runs-on }}${{ env.SCCACHE_KEY_SUFFIX }}-${{ github.run_id }} | ||
| key-prefix: sccache-sqlcatalog-${{ matrix.runs-on }} |
Comment on lines
+62
to
+65
| - name: Save sccache | ||
| uses: ./.github/actions/save-sccache | ||
| with: | ||
| path: ${{ github.workspace }}/.sccache | ||
| key: sccache-sanitizer-ubuntu-${{ github.run_id }} | ||
| key-prefix: sccache-sanitizer-ubuntu |
Comment on lines
+124
to
+127
| - name: Save sccache | ||
| uses: ./.github/actions/save-sccache | ||
| with: | ||
| path: ${{ github.workspace }}/.sccache | ||
| key: sccache-aws-${{ matrix.runs-on }}-bundle${{ matrix.bundle_awssdk }}-s3${{ matrix.s3 }}-sigv4${{ matrix.sigv4 }}-${{ github.run_id }} | ||
| key-prefix: sccache-aws-${{ matrix.runs-on }}-bundle${{ matrix.bundle_awssdk }}-s3${{ matrix.s3 }}-sigv4${{ matrix.sigv4 }} |
Comment on lines
+77
to
+80
| - name: Save sccache | ||
| uses: ./.github/actions/save-sccache | ||
| with: | ||
| path: ${{ github.workspace }}/.sccache | ||
| key: sccache-cpp-linter-ubuntu-${{ github.run_id }} | ||
| key-prefix: sccache-cpp-linter-ubuntu |
wgtmac
reviewed
Jul 23, 2026
| using: composite | ||
| steps: | ||
| - name: Show sccache stats | ||
| if: always() |
Member
There was a problem hiding this comment.
always() only applies after the composite action starts. The call sites still default to success(), so this action is skipped after a failed build and stats are not shown. Could we add if: always() to the call sites and keep cache saving success-gated?
Contributor
Author
There was a problem hiding this comment.
Definitely! I've updated the jobs to propagate the status of the job from the call sites then gate the save on that status but leave the show sccache stats ungated.
abnobdoss
force-pushed
the
ci/share-sccache-setup
branch
from
July 23, 2026 23:36
493a9e1 to
05aa905
Compare
Contributor
Author
|
Hi @wgtmac, thank you for reviewing! I've rebased and updated the composite action gating per the feedback. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add two composite actions for sccache:
.github/actions/setup-sccachesets the sccache environment, restores the cache, resolves the MSVC compiler version on Windows, and startsmozilla-actions/sccache-action..github/actions/save-sccacheshows sccache stats for every run and saves the cache only on main branch runs.The Test, SQL Catalog, sanitizer, C++ linter, and AWS workflows now pass only their cache key prefix at each call site.
This is part of the CI work discussed in #799.
Notes
The existing cache key shapes are preserved for the current CMake workflow legs. The Meson Windows leg now also gets the MSVC-version key suffix, matching the other Windows sccache legs and avoids cache misses after runner image compiler updates.
The MSVC version source also changes for the existing Windows legs (test and SQL catalog): the inline
cl.exebanner parse is replaced by readingVCToolsVersion.default.txtvia vswhere, which is more reliable and does not requirecl.exeon PATH. This produces a different version format (e.g.14.44.35217vs19.44.35217.1), so all Windows caches will miss once and reseed on the first post-merge main run.Third-party actions stay pinned inside composite
action.ymlfiles, which are covered by the ASF allowlist checker glob.Validation
.github/**, so this PR does not exercise that workflow until a source-touching run or post-merge.